Situated Interaction in a Multilingual Spoken Information Access Framework

The paper describes aspects of situated interaction when a humanoid robot uses the WikiTalk system as a spoken language dialogue interface. WikiTalk is a speech-based open-domain information access system that enables the user to move around Wikipedia from topic to topic and have chunks of interesting articles read out aloud. The interactions with the robot are situated: they take place in a particular context and are driven according to the user’s interest and focus of attention. The interactions are also multimodal as both user and robot extend their communicative repertoire with multimodal signals. The robot uses face-tracking, nodding and gesturing to support interaction management and the presentation of new information to the partner, while the user speaks, moves, and can touch the robot to interrupt it.